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DETAILED ACTION 
Oath/Declaration 

1. Oath/Declaration submitted by applicant on 08/12/2003 is accepted. 

Drawings 

2. The drawings submitted by applicant on 01/16/2004 are accepted. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only If the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

4. Claims 1 , 3 - 7, 1 2 - 1 3, 1 8 and 22 - 23 are rejected under 35 U.S.C. 1 02(e) as 
being anticipated by Broemmelsiek (US 2002/0030741). 

Regarding claim 1; Broemmelsiek 741 discloses, a method for perfomiing 
Surveillance (i.e. fig. 1, page 1, paragraph 0002 and page 2, paragraphs 0023 and 
0025) comprising: monitoring an indicia of manual imaging sensor control (i.e. page 2, 
paragraph 0022, lines 6-8 and page 3. paragraph 0028, where discloses user input 
device by which user can control the camera manually, e.g. the indicia indicates that, 
manual control is being used), and switching to automatic imaging sensor control when 
the indicia indicates that manual imaging sensor control is not being used (i.e. page, 2, 
paragraphs 0022 and 0025, discloses the camera processing system can be performed 
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automatically or manually, which implies that, when the camera is in automatic control 
mode, the manual imaging control would be disabled, e.g. the indicia indicates that 
manual control is not being used at this time) and switching to manual imaging sensor 
control when the indicia indicates that manual imaging sensor control is being used 
(please see; page, 2, paragraphs 0022 and 0025, where discloses the camera 
processing system can be perfonned automatically or manually, which implies that, 
when the camera is manually controlled through a user input device "page 3, paragraph 
0028", e.g. indicia indicates that manual imaging sensor control is being used and the 
automatic control of the camera would be disabled at this time and the camera is being 
controlled based on the user input commands), and determining imaging sensor 
coordinates for at least one imaging sensor (i.e. page 2, left column, lines 5-8 and 
page 3, paragraph 0035, lines 1-4, where discloses coordinate position of the movable 
video camera) and mapping the imaging sensor coordinates to image pixel coordinates 
(i.e. page 3, paragraphs 0034 and 0037, where mapping of camera coordinates 
(position) conresponds to image pixel coordinates) and processing the image pixel 
coordinates to detenmine information regarding object within a scene that is captured by 
at least one imaging sensor (i.e. page 2, paragraph 0025 and page 3, paragraphs 0034 
- 0037, where discloses processing of the image and mapping coordinates which 
corresponds to image pixel coordinates to determine infomnation regarding object; e.g. 
motion of objects within the field of view and based on the motion of the object, controls 
the camera to hold the object of interest in a field of view of the camera). 
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Regarding claim 3, Broemmelsiek 741 discloses, wlierein tlie orientation of the 
imaging sensor comprises pan, tilt and zoom (please see; page 4, paragraph 0039, PTZ 
coordinates). 

Regarding claim 4, Broemmelsiek 741 discloses, detecting object motion within 
the scene (i.e. page 2. paragraph, 0025, where discloses process of automatic moving 
object detection, by thresholding techniques to detect motion of the moving object). 

Regarding claim 5, Broemmelsiek 741 discloses, upon detecting object motion 
during automatic imaging sensor control (i.e. page, 2, paragraphs 0025, where 
discloses automatic detection of moving object) switch to manual imaging sensor 
control (please see; page, 2, paragraphs 0025 and page 3, paragraph 0028, where 
discloses automatically detecting moving objects, and after automatic detection of 
moving object the user input commands can be accepted by the system and used to 
control the camera manually, e.g. upon automatic detection of moving object, user input 
commands can be accepted to manually control the imaging sensor). 

Regarding claim 6, Broemmelsiek 741 discloses, a method for performing 
Surveillance (i.e. fig. 1 , pages 1-2; paragraphs 0002, 0023 and 0025) comprising: 
pointing an imaging sensor at a location in a scene (see; fig. 1, field of view of the 
camera 12 and camera controller 14, page 2, paragraph 0023 and page 4, paragraph 
0039), and detecting a moving object within the scene (i.e. page 2, paragraph 0025, 
where discloses detection of moving object), and controlling pan, tilt and zoom functions 
of the imaging sensor to automatically track the moving object (please see; page 2, 
paragraph 0022 and page 4, paragraph 0039, where discloses detection of object 
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automatically and further states that, automatically or manually direct the camera's line- 
of-sight (e.g. PTZ) to track the object, which implies that the system is capable of 
controlling the camera (PTZ) to track the object in automatic mode). 

Regarding claim 7, Broemmelsiek 741 discloses, manually directing the imaging 
sensor to image the location (see; fig. 1 , field of view of the camera 12 and camera 
controller 14, page 3, paragraph 0028, where discloses manually controlling the camera 
by user input). 

Regarding claim 12, Broemmelsiek 741 discloses, wherein the location is a 
region of the scene that is imaged by the imaging sensor (i.e. fig. 1 , field of view of 
camera 12) and further comprising establishing a sensitivity level for processing 
information within the region (please see; page 2, paragraph 0025, where discloses 
applying thresholding techniques (e.g. sensitivity level) for processing the information to 
detect the motion of the object within the scene/field of view of the imaging 
sensor/camera). 

Regarding claim 13, Broemmelsiek 741 discloses, wherein the sensitivity level 
establishes a motion detection sensitivity level (i.e. page 2, paragraph 0025, where 
discloses applying thresholding techniques (e.g. sensitivity level) to establishes a 
motion detection, motion of the object within the field of view of the imaging 
sensor/camera). 

Regarding claim 18, Broemmelsiek 741 discloses, an imaging sensor control 
module for pointing an imaging sensor at a location in a scene (see; fig. 2, input device 
20 and controller 14, page 3, paragraph 0028, lines 1 - 3), and an image processor for 
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detecting a moving object within tlie scene (i.e. figs. 1-2, camera processing 22, page 
2, paragraph 0025, where discloses processing of the image captured by the camera to 
detect moving object within the scene/field of view of the camera) and a controller for 
controlling pan, tilt and zoom functions of the imaging sensor to automatically track the 
moving object (please see; page 2, paragraph 0022, where discloses automatically 
select a moving object and direct the camera's line-of-sight (e.g. PTZ) to track the 
moving object). 

Regarding claim 22, Broemmelsiek 741 discloses, wherein the location is a 
region of the scene that is imaged by the imaging sensor (i.e. fig. 1 , field of view of 
camera12, which consider as region of the scene that is imaged by camera 12) and 
further comprising establishing a sensitivity level for processing information within the 
region (please see; page 2, paragraph 0025, where discloses applying thresholding 
techniques (e.g. sensitivity level) for processing the information to detect the motion of 
the object within the scene/field of view of the imaging sensor/camera). 

Regarding claim 23, Broemmelsiek 741 discloses, wherein the sensitivity level 
establishes a motion detection sensitivity level (i.e. page 2, paragraph 0025, where 
discloses applying thresholding techniques (e.g. sensitivity level) to establishes a 
motion detection, motion of the object within the field of view of the imaging 
sensor/camera). 

5. Claims 6-11,14-21 and 24 - 27 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Sengupta et al. (US 6,359,647). 

Regarding claim 6, Sengupta '647 discloses, a method for performing 
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Surveillance (i.e. fig. 1 . col. 3, lines 9-37 and cols. 4-5, lines 46-18) comprising: 
pointing an imaging sensor at a location in a scene (please see; fig. 1 , field of view of 
the adjustable camerias 101 and 102 and controller 130, col. 3, lines 9 - 22), and 
detecting a moving object within the scene (see; fig. 1, location determinator 140 and 
figure tracking 144, col. 3, lines 54 - 55, where detects the moving object and 
determines the location of the moving object within the camera's field of view), and 
controlling pan, tilt and zoom functions of the imaging sensor to automatically track the 
moving object (i.e. fig. 1, adjustable cameras 101 and 102 and controller 130, col. 3, 
lines 25 - 28, where discloses controlling pan, tilt and zoom functions of the selected 
imaging sensor, and col. 2, lines 47 - 50 and col. 3, lines 66 - 67, to automatically track 
the moving object). 

Regarding claim 7, Sengupta '647 discloses, manually directing the imaging 
sensor to image the location (please see; col. 3, lines 23 - 28 and col. 4, lines 20 - 25, 
operator manually controlling imaging sensor via operator station 1 70 and controller 
130). 

Regarding claim 8, Sengupta '647 discloses, wherein the pointing step further 
comprises automatically scanning the imaging sensor across the scene to image the 
location; and continue scanning the scene until the moving object is detected (i.e. col. 4, 
lines 8-13 and col. 13, lines 38 - 65, where indicates that the automated means can 
be applied (e.g. imaging sensor) to image the scene and continually monitor (e.g. scan) 
the scene to detect the moving object and reports the moving object location). 
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Regarding claim 9, Sengupta '647 discloses, deriving a latitude, longitude and 
altitude of the moving object based upon the pan. tilt and zoom parameters of the 
imaging (please see; col. 5. lines 19-67, which determines location of the object in 
three dimensional coordinates; with respect to the cameras pan. tilt and zoom; it is 
noted that, tilt angle of the camera referred to as the latitude of the object/target, and 
pan position consider as longitude, and the third dimension is the altitude or height of 
the object in three dimensional coordinates, which defines a set of object coordinates 
for object(s) in the field of view of the camera). 

Regarding claim 10, Sengupta '647 discloses, predicting a sight line between the 
imaging sensor, and the moving object, and using the sight line to optimally image the 
moving object (i.e. col. 2, lines 27 - 36. col. 4. lines 27 - 45, col. 6. lines 15-39, where 
discloses predicting location of the moving object based on camera's line of sight and 
the moving object). 

Regarding claim 11. Sengupta '647 discloses, a plurality of imaging sensors and 
using the sight line to select an imaging sensor from the plurality of imaging sensor that 
provides an optimal view of the moving object (see; col. 4, lines 27 - 45 and col. 7, lines 
57 - col. 8, lines 31 , where indicates using the line of sight of camera and actual field of 
view to adjust/control and select the camera to maintain the target figure in the center of 
image from camera (e.g. optimal view of the object)). 

Regarding claim 14, Sengupta '647 discloses, wherein the information produced 
by the imaging sensor is displayed in combination with a zone map that depicts the 
scene (please see; figs. 3a - 3c and 5C, where shows Information produced by the 
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imaging sensor "fig. 5c, target P and cameras field of view 581 and 582, and line of 
sight 580" in combination with the zone map). 

Regarding claim 15, Sengupta '647 discloses, wherein coordinates of locations in 
the zone map are mapped to pan/tilt/zoom parameters using a look-up table (please 
see; col. 3, lines 38 - 53 and col. 4, lines 46 - col. 5, lines 18; that is in order to effect 
this automatic selection of cameras, the camera handoff system uses the infomiation 
contained in the data base 160 (which consider as look-up table) to control/adjust the 
camera(s), pan tilt and zoom). 

Regarding claim 16, Sengupta '647 discloses, further comprising combining 
information from a plurality of imaging sensors with the zone map (please see; figs. 3a - 
3c, where shows zone map in combination with the imaging sensors locations, col. 4, 
line 46 - col. 5, lines 18). 

Regarding claim 17, Sengupta '647 discloses, wherein a plurality of imaging 
sensors image the scene (i.e. fig. 1, cameras 101-103 and controller 130) and the 
altitude, longitude and altitude of the moving object are used to handoff Imaging of one 
imaging sensor to another imaging sensor (please see; col. 3, lines 38 - 65 and col. 4, 
lines 35 - 45, where discloses, switching from one camera to another based on the 
camera field of view and object location by using 3D coordinate, it is noted that 
"determininjg location of the object in three dimensional coordinates as discussed earlier 
in claim 9 above, by using latitude, longitude and altitude of the moving object based 
upon the pan, tilt and zoom parameters of the imaging sensor; tilt angle of the camera 

« 

referred to as the latitude of the object/target, and pan position of the camera consider 
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as longitude, and the third dimension is the altitude or height of the object in three 
dimensional coordinates, which defines a set of object coordinates for object(s) in the 
field of view of the camera, to determine the object location in 3D coordinates"). 

Regarding claim 18, Sengupta '647 discloses, an imaging sensor control module 
for pointing an imaging sensor at a location in a scene (please see; fig. 1 , field of view of 
the adjustable cameras 101 and 102 and controller 130, col. 3, lines 9 - 22) and an 
image processor for detecting a moving object within the scene (see; figs. 1 , elements 
140-142 and user station 170, for processing the image captured by the cameras to 
detect the moving target/object within the field of view of the camera, col. 1 , lines 42 - 
45, and col. 5, lines 54 - 61) and a controller for controlling pan, tilt and zoom functions 
of the imaging sensor (see; fig. 1, adjustable cameras 101 and 102, controller 130 and 
col. 3, lines 25 - 28) to automatically track the moving object (i.e. fig. 1 , figure tracking 
system, col. 2, lines 47 - 50 and col. 3, lines 66 - 67, where discloses figure tracking 
system automatically track the figure/object). 

Regarding claim 19, Sengupta '647 discloses, wherein the image processor 
derives a latitude, longitude and altitude of the moving object based upon pan, tilt and 
zoom parameters of the imaging sensor (please see; col. 5, lines 19-67, which 
detenmlnes location of the object in three dimensional coordinates; with respect to the 
cameras pan, tilt and zoom; it is noted that, filt angle of the camera referred to as the 
latitude of the object/target, and pan position consider as longitude and the third 
dimension is the altitude or height of the object in three dimensional coordinates, which 
defines a set of object coordinates for object(s) in the field of view of the camera). 
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Regarding claim 20, Sengupta '647 discloses, wherein the image processor 
predicts a sight line between the imaging sensor and the moving object and uses the 
sight line to enable the Imaging sensor control module to point the imagines sensor to 
optimally image the moving object (see; col. 2, lines 27 - 36, col. 4, lines 27 - 45 and 
col. 6, lines 15-39, where Indicates using the line of sight of camera and actual field of 
view to adjust/control the camera to maintain the target figure in the center of Image 
from camera (e.g. optimal image the moving object)). 

Regarding claim 21, Sengupta '647 discloses, a plurality of imaging sensors and 
using the sight line to select an imaging sensor from the plurality of imaging sensor that 
provides an optimal view of the moving object (please see; col. 4, lines 27 - 45 and col. 
7, lines 57 - col. 8, lines 31 , where indicates using the line of sight of camera and actual 
field of view to adjust/control and select the camera to maintain the target figure in the 
center of image from camera (e.g. optimal view of the object)). 

Regarding claim 24, Sengupta '647 discloses, wherein the information produced 
by the imaging sensor is displayed in combination with a zone map that depicts the 
scene (i.e. fig. 5c, where shows information produced by the imaging sensor "target P 
and cameras field of view 581 and 582, and line of sight 580" in combination with the 
zone map). 

Regarding claim 25, Sengupta '647 discloses, wherein the information produced 
by the imaging sensor is displayed in combination with a zone map that depicts the 
scene (please see; figs. 3a - 3c and 5C, where shows information produced by the 
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imaging sensor "fig. 5c, target P and cameras field of view 581 and 582, and line of 
sight 580" in combination with the zone map). 

Regarding claim 26, Sengupta '647 discloses, further comprising combining 
infomnation from a plurality of imaging sensors with the zone map (i.e. figs. 3a - 3c and 
5C, where shows zone map in combination with the imaging sensors information, col. 4, 
line 46 - col. 5, lines 18). 

Regarding claim 27, Sengupta '647 discloses, wherein a plurality of imaging 
sensors image the scene (i.e. fig. 1, cameras 101-103 and controller 130) and the 
altitude, longitude and altitude of the moving object are used to handoff imaging of one 
imaging sensor to another imaging sensor (please see; col. 3, lines 38 - 65 and col. 4, 
lines 35 - 45, where discloses, switching from one camera to another based on the 
camera field of view and object location by using 3D coordinate, it is noted that 
"determining location of the object in three dimensional coordinates as discussed earlier 
in claim 9 above, by using latitude, longitude and altitude of the moving object based 
upon the pan, tilt and zoom parameters of the Imaging sensor; tilt angle of the camera 
referred to as the latitude of the object/target, and pan position of the camera consider 
as longitude, and the third dimension is the altitude or height of the object in three 
dimensional coordinates, which defines a set of object coordinates for object(s) in the 
field of view of the camera, to determine the object location in 3D coordinates"). 

Claim Rejections - 35 USC § 103 
6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Broemmelsiek (US 2002/0030741 ) in view of Cohen-Solal et al. (US 7,173,650). 

Regarding claim 2, Broemmelsiek '741 teaches, user input device, which can be 
used for control of the camera (I.e. page 3, paragraph 0028, lines 1 - 3). 

Broemmelsiek 741 is silent in regards to "manipulating a joystick" to control an 
Orientation of an imaging sensor and the indicia comprises movement of the joystick. 

Cohen '650 in the same field of automated video tracking system (I.e. col. 4, jines 
30 - 33 and col. 6, lines 1 - 9) teaches, "manipulating a joystick" to control camera 
orientation (pan, tilt and zoom) by moving the joystick. 

In view of the above, it would have been obvious to one having ordinary skill in 
the art at the time of the invention was made to provide a joystick as taught by Cohen 
'650 (i.e. col. 4, lines 30 - 33 and col. 6, lines 1 - 9) as an input device (I.e. fig. 2, 
element 20) in Broemmelsiek's object tracking system, to manually control camera 
orientation (pan, tilt and zoom) for tracking the object, as suggested by Cohen '650 (I.e. 
col. 6, lines 2-7). 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

Pretzer et al. (US 2003/01 031 39 Al ), System and Method for tracking objects and 
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obscuring fields of view under video surveillance. 

Cavallaro et al. (US 6,133,946), System for determining the Position of an Object. 
Gero (US 2004/0046938 Al), Automatic and manual lens focusing system with visual 
matching for motion picture camera. 

Contact 

9. Any inquiry concerning this communication or earlier communications 
from the examiner should be directed to Behrooz Senfi whose telephone 
number is (571) 272-7339. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. 

Hand-delivered responses should be brought to Randolph Building, 401 Dulany 
Street, Alexandria, Va. 22314. 

. Any inquiry of a general nature or relative to the status of the application or 
proceeding should be directed to the Technology Center 2600 Customer Service Office 
whose telephone number is (571) 272-6000, 
Or faxed to: 

(571)273-8300 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status infomiation for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
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more information about tlie PAIR system, see littp://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-21 7-91 97 (toll-free). 
B.M.S. 




